I. Field of the Invention
This invention relates to spectrophotometers capable of measuring electromagnetic radiation, such as infrared, near-infrared, ultraviolet, visible, Ramon spectra, Rayleigh scattering, and the like. More specifically, this invention relates to calibration techniques useful for predicting chemical or physical properties based on observed spectral data, preferably data obtained by means of a spectrophotometer. Still more preferably, this invention embodies near-infrared calibration equations suitable for making predictions or estimates of physical or chemical properties such as octane.
II. Prior Art Summary
In J. Sci. Food Agric., 34(1983), p. 1441-1443, B. G. Osborne directly transfers a calibration equation from one instrument to another.
Use of Indirect Multivariate Calibration for Quality Control of Agricultural Products by Near-Infrared Spectroscopy, G. Puchwein and A. Eibelhuber (Mikrochim. Acta (Wien), 1986 II, 43-51) teach a computer program for calibrating an instrument. The program picks relevant frequencies for spectral data for part of a set of samples with which to calibrate an instrument and then validates with respect to the remaining samples.
Near-Infrared Limitations to Silicon Photodetector Self-Calibration, J. M. Palmer (SPIE Vol. 499, Optical Radiation Measurements [1984], p. 7-14) teaches self-calibrating silicon photodiode detectors.
A Simple Internal Modulation Technique for the Spectral Calibration of Circular Variable Filter Spectrometers in the Near-Infrared, Chandrasekhar, Ashok, Bhatt, and Manian (Infrared Physics, Volume 24, No. 6, 1984, p. 571-572) teach the wavelength calibration of telescopes attached to near-infrared instruments for astronomical studies using the frequency dependent signal of a low-pressure mercury arc lamp.
Mark and Workman (Spectroscopy, 311), 28) developed a computer algorithm to find isonumeric wavelengths. Isonumeric wavelengths are calibration wavelengths which yield the same absorbance readout on a master NIR and a slave NIR. These wavelengths therefore do not change substantially from one instrument to another.
Heckman, Diffee, and Milhous (Analytica Chemica Acta, 192(1987), 197) teach a computer program that transfers calibration equations by computing slope and bias corrections for each wavelength in the equation. This method of slope and bias corrections is, in fact, used in Example 3 of this application. In U.S. Pat. No. 4,761,522 R. D. Rosenthaul claims a poly-tetrafluoroethylene standard for near-infrared and processes for making that standard. That standard, however, is not used in calibration transfer from one instrument to another.
B. G. Osborne, in an article entitled, Calibration of Instrument for Near-Infrared Spectroscopy, (48 Spectroscopy, Vol. 4 No. 4, p. 48-55). Osborne discloses in detail the steps for establishing simple correlations by multiple linear regression and also the use of derivatives of linear terms. Validation of calibration curves are discussed; specifically how particular wavelengths are chosen to establish the terms in the multiple linear regression equations, along with the statistical significance of a particular calibration sample. This method can be utilized in this invention for the purpose of defining the calibration equations of the calibrated instrument for a range of values within which useful predictions can be made, i.e., the range of interest.
Still other methods for determining calibration equations are those discussed in Analytical Chemistry Journal Vol. 60, pages 1193-1202 (1988) by David M. Haaland and Edward V. Thomas. The principles of inverse least squares, partial least squares, classical least squares, and principal component regression analysis are disclosed. More specifically, for purposes of the following discussion, assume: capital letters in boldface identify matrices, small letters in boldface identify column vectors, small letters in italics identify scalers, and any primes used in conjunction with boldface letters signify a transposition.
Using the notation just discussed and the following assumptions: a Beer's law model for m, calibration standards containing l, chemical components with a spectra of n, digitized absorbances, the following equation in matrix notation holds: EQU A=CK+E.sub.A Equation ( 1)
where A is the m.times.n matrix of calibration spectra, C is the m.times.l matrix of component concentrations, K is the l.times.n matrix of absorptivity--path length products, and E.sub.A is the model. K then represents the matrix of pure-component spectra at unit concentration and unit path length. The classical least-squares solution to Equation 1 during calibration is: EQU K=(C'C).sup.-1 C'A Equation (2)
where K indicates the least-squares estimate of K with the sum of squared spectral errors being minimized. During prediction, the least-squares solution for the vector of unknown component concentrations, c, is: EQU c=( K K').sup.-1 Ka Equation (3)
where a is the spectrum of the unknown sample and K is from Equation 2.
Equation 1 shows that CLS can be considered a factor analysis method since the spectral matrix A is represented as the product of two smaller matrices C and K. The pure-component spectra (rows of K) are the factor loadings (also called loading vectors) and the chemical concentration (elements in C) are the factors (or scores). This model changes the representation of the calibration spectra into a new coordinate system with the new coordinates being the l pure-component spectra rather than the n spectral frequencies. Although this coordinate system is not necessarily orthogonal, it has the advantage that the l spectral intensities for each mixture in this new coordinate system of pure-component spectra are the elements of C; i.e., the intensities in the new coordinate system are the component concentrations. This is clear when one considers that the component concentrations represent the amount (or intensities) of the pure-component spectra which make up any given mixture spectrum.
Since CLS is a full-spectrum method, it can provide significant improvements in precision over methods that are restricted to a small number of frequencies, allow simultaneous fitting of spectral base lines, and make available for examination and interpretation least-squares estimated pure-component spectra and full-spectrum residuals.
The inverse-least square ("ILS") method assumes concentration is a function of absorbance. An inverse Beer's law model for m, calibration standards with spectra of n digitized absorbances is given by: EQU C=AP+E.sub.c Equation ( 4)
where C and A are as before, P is the n.times.l matrix of the unknown calibration coefficients relating the l component concentrations to the spectral intensities, and E.sub.c is the m.times.l vector of random concentration errors or residuals that are not fit by the model. Since model error is presumed to be error in the component concentrations, this method minimizes the squared errors in concentrations during calibration. The inverse representation of Beer's law has the significant advantage in that the analysis based on this model is invariant with respect to the number of chemical components, l, included in the analysis. If it is assumed that the elements in different columns of Ec are independent, an identical analysis for each individual analyte can be obtained by considering the reduced model for one component: EQU c=Ap+e.sub.c Equation ( 5).
Here is the m.times.1 vector of concentrations of the analyte of interest in the m calibration samples, p is then an n.times.1 vector of calibration coefficients, and e.sub.c is the m.times.1 vector of concentration residuals not fit by the model.
During calibration, the least-squares solution for p in eq 5 is: EQU p=(A'A).sup.-1 A'c Equation (6).
During prediction, the solution for the analyte concentration in the unknown sample is simply: EQU c=a' p Equation (7).
This means a quantitative spectral analysis can be performed even if the concentration of only one component is known in the calibration mixtures. The components not included in the analysis must be present and implicitly modeled during calibration. The above capability of the ILS method has resulted in it being used for near-infrared analysis ("NIRA") methods.
Partially squares ("PLS") and principal component regression ("PCR") are both factor analysis methods with many of the advantages of the CLS method. Using the spectral decomposition notation of Lindberg, et al, Anal. Chem. 1983, 55, 643, the calibration equation can be represented for either a principal component analysis ("PCA") or PLS model as follows: EQU A=TB+E.sub.A Equation ( 8)
where B is a h.times.n matrix with the rows of B being the new PLS or PCA basis set of h full-spectrum vectors, often called loading vectors or loading spectra. T is an m.times.h matrix of intensities (or scores) in the new coordinate system of the h PLS or PCA loading vectors for the m sample spectra. In PCA the rows of B are eigenvectors of A'A, and the columns of T are proportional to the eigenvectors of AA'. E.sub.A is now the m.times.n matrix of spectral residuals not fit by the optimal PLS or PCR model. The analogy between eq 8 of PLS or PCA and eq 1 for CLS is quite clear since both equations involve the decomposition of A into the product of two smaller matrices. However, now rather than the basis vectors being the pure-component spectra, they are the loading vectors generated by the PLS or PCA algorithms. The intensities in the new coordinate system are no longer the concentrations as they were in CLS, but they can be modeled as linearly related to concentrations as shown later. The new basis set of full-spectrum loading vectors is composed of linear combinations of the original calibration spectra. The amounts (i.e., intensities) of each of the loading vectors which are required to reconstruct each calibration spectrum are the scores.
In general, in a noise-free system only a small number of the full-spectrum basis vectors are required to represent the calibration spectra (A). When the rank of A which is important for concentration prediction is r, then the optimal PLS or PCR model in eq 8 will have the dimension h equal to r. In general r&lt;m and r&lt;n, in which case PLS and PCA will have reduced the number of intensities (n) of each spectrum in the spectral matrix A to a small number of intensities (r) in the new coordinate system of the loading vectors. This data compression step also reduces the noise (32) since noise is distributed throughout all loading vectors while the true spectral variation is generally concentrated in the early loading vectors.
The spectral intensities (T) in the new coordinate system can be related to concentrations with a separate inverse least-squares analysis using a model similar to eq 5. However, rather than solving eq 5 by least-squares methods with the problem of calculating (A'A).sup.-1, we solve the following set of equations by least squares: EQU c=Tv+e.sub.c Equation ( 9)
where v is the h.times.1 vector of coefficients relating the scores to the concentrations and T is the matrix of scores (intensities) from the PLS or PCA spectral decomposition in eq 8.
In conclusion, PLS and PCR are both involved in an inverse lease-squares step (PCR is simply PCA followed by the separate regression step for the model given in equation nine).
Although PCA and PLS are similar, the methods to accomplish the goals of spectral decomposition and concentration prediction are different. Both methods as implemented here involved stepwise algorithms which calculate the B and T matrices one vector at a time until the desired model has been obtained. In general, different T and B matrices are generated by the PLS and PCA methods. In PCA, the columns of T are orthogonal and the rows of B are orthogonal while in the version of PLS presented here only the columns of T are orthogonal. The PCA algorithm used here is the NIPALS (nonlinear iterative partial least-squares) algorithm developed by Wold, H. Multivariate Analysis; Krishnalah, P. R., Ed.; Academic: New York, 1966; page 391. NIPALS is an efficient iterative algorithm which extracts the full-spectrum loading vectors (eigenvectors of A'A) from the spectra in the order of their contribution to the variance in the calibration spectra. After the first loading vector has been determined, it is removed from each calibration spectrum, and the process is repeated until the desired number of loading vectors has been calculated. The potential problem with PCR is that the loading vectors which best represent the spectral data may not be optimal for concentration prediction. Therefore, it would be desirable to derive loading vectors so that more predictive information is placed in the first factors. The PLS algorithm presented here is a modification of the NIPALS algorithm, and it achieves the above goal by using concentration information to obtain the decomposition of the spectral matrix A in eq 8. Concentration-dependent loading vectors are generated (B) and the computed scores (T) are then related to the concentrations or concentration residuals after each loading vector is calculated. Therefore, in principle, greater predictive ability is forced into the early PLS loading vectors.
Liquid Absorption Standards For Ultraviolet, Visible, and Near-Infrared Spectrophotometry, R. G. Martinek, J. Amer. Med. Technol. July-August, 1978, on p. 210-216. Liquid absorbance standards are shown to have the following desirable characteristics: broad absorption peaks; wide wavelength range; high molar absorptivity; stability in solution; readily definable specifications of purity; and minimal spectral temperature coefficients. The purpose of using such standards is to permit calculation of absolute absorbance based upon the observed absorbance and the application of an instrument correction factor.
The Use of Statistics In Calibrating and Validating Near Infrared On-Line Analyzers, Bruce Thompson, ISA, 1989 Paper #88-0116, uses near-infrared for both process control and statistical process control.
Calibration of Instruments for Near-Infrared Spectroscopy, B. G. Osborne (Spectroscopy, 4(4), p. 48-50) teaches the need for multiplicative scattering correction for solid samples due to particle-size differences in solids and computer transfer of complex equations. Osborne discusses procedures for developing calibration equations of particular relevance and usefulness in the present invention. The selection of appropriate wavelengths and samples are discussed and relevant articles cited further explaining the techniques employed. The transfer of calibration equations from one instrument to another are disclosed to require, in the case of most modern instruments, only a change in intercept value, and computerized procedures allow for differences between spectral data generated by different instruments. These computerized procedures are not taught in detail, but are likely to involve transforming the spectral data of one instrument to be consistent with that of another instrument. This is to be distinguished from the method involved in the present invention, which instead of transforming the spectral data, the relevant coefficients in the calibration equation for the second instrument are changed from those found in the calibration equation of the first instrument in accordance with a unique procedure discussed in this specification.
In the inventor's doctoral dissertation, A Chemometric Analysis of a Magnetic Water Treatment Device available from University Microfilms International, Dissertation Information Service, order number 8919928, there is a discussion of linear regression analysis (simple and multiple) for a linear regression model, and data transformations which can transform non-linear types of data into data which is appropriate for linear regression modeling. Definitions are shown which establish the degree of reliability that the value of one variable (the dependent variable) can be estimated on the basis of values for other variables (the independent variables). In general, the methods for determining regression constants (or coefficients) for the calibration equations discussed in the inventor's doctoral thesis are useful in establishing the calibration equation for the first or reference instrument in the present invention.
A New Approach to Generating Transferable Calibrations for Quantitative Near Infrared Spectroscopy, H. Mark and J. Workman, Spectroscopy Volume 3, No. 11, p. 28-36, shows that instrument variability makes transfer of calibrations between instruments difficult. However, by selecting the wavelengths and corresponding spectral properties for such wavelengths to meet certain criterion, many of the variations between instruments will not change significantly the calibration equation from one instrument for that calibration equation which would work satisfactorily in another instrument. Not disclosed is how to revise one calibration equation from one instrument to be suitable for use in another instrument where there is a significant change in the calibration equation suitable for the second instrument.
Tobacco constituents to Tilting Filter Instruments, R. A. Heckman, J. T. Diffee, L. A. Milhous, Analytica Chemica Acta, 192(1987) p. 197-203, teach transferring a near-infrared monochromatic calibration for tobacco constituents to a tilting-filter instrument in order to utilize the experimental calibration equation for a sensitive but not rugged laboratory instrument to analyze data in a rugged production oriented instrument to predict properties such as moisture, glycerin, propylene glycol, nicotine, and reducing sugar. A program called MTRAN cannot be used without difficulty unless care was exercised to insure that the tilting-filter instrument is properly configured with filters so as to accommodate the wavelengths in the primary or master calibration equation for the laboratory instrument. Consequently, before the primary calibration can be transformed, the monochromatic data files are compressed to simulate a tilting-filter instrument. The resulting data file is used to re-establish the calibration equation. With the foregoing changes, CALTRAN, a program formerly used only for calibration transfer between filter instruments, could be used to effect transfer from a near-infrared monochromatic instrument to a tilting filter instrument.
Typically, the prior art seeks to make changes in the observed spectra (or mathematical transform thereof) produced in one instrument conform to those produced by a calibrated reference instrument. Only after the spectra from an uncalibrated instrument is brought into coincidence with that spectra from a calibrated instrument did one seek to transfer a calibration equation from one instrument to another. Our experience has been that this tends to introduce errors, uncertainty, and numerous numerical difficulties which reduce the ease and precision that is made possible by the invention of this specification.
U.S. Pat. No. 4,963,745 to Maggard entitled Octane Measuring Process and Device discloses how to initially establish a calibration equation for a reference instrument. Specifically disclosed is that spectral features, such as second derivatives, of infrared spectral data at methyne band frequencies, in the range of about 1200 to 1236 nanometers (nm), were highly correlatable to octane values for gasoline-like substances. Numerous patents and articles are cited within the text of the specification, as references cited, and as other publications, all of which including those expressly cited are expressly incorporated herein by reference.
Most methods using calibration samples, seek to place one instrument in substantially the same condition as that of another so as to produce substantially identical spectral responses. With substantially identical spectral outputs, identical inferences are then drawn.
An example of a linear calibration equation is one of the form Y.sub.c.sup.i =A.sup.i +.SIGMA..sub.j B.sub.j.sup.i X.sub.jc.sup.i where a Y.sub.c.sup.i is the predicted value of some physical or chemical property, such as octane values for a gasoline, C, of a set called sample-c. As discussed in Example 7, 226 gasoline samples were measured to determine A.sup.i, and each B.sub.jc.sup.i for j=1 to 3, to obtain a multiple regression equation suitable for predicting pump octane values of gasoline samples, using a first instrument; instrument-i, the reference instrument. Pump octane is the average of ASTM methods D2699 and D2700, and is commonly known as (R+M)/2 octane. The inferred reproducibility of this test method is a 95% confidence limit of .+-.0.71 octane numbers over the octane range of 84-94 pump octane. From the second instrument, instrument-k, while in transflectance mode, gasoline absorbances at 1196, 1220, and 1236 nanometers (nm) were used to determine the values for the second derivative of an absorbance spectra, the spectral features, X.sub.jc.sup.k, in the linear calibration equation for inferring a pump octane. The standard error of prediction ("SEP") is the statistical parameter that equals the square root of the mean residual variance of unknown samples.